Clickable camera window

ABSTRACT

Embodiments are directed to receiving, by a control station, an input including a command for re-location of a vehicle from a first location to a second location, the input identifying the second location, determining, by the control station, at least one of a distance, a direction, an altitude, and a latitude and longitude for the vehicle to travel from the first location to the second location, and transmitting, by the control station, the command and the at least one of a distance, a direction, an altitude, and a latitude and longitude to the vehicle.

BACKGROUND

When controlling an aircraft from a ground control station (GCS) oralternate control device, it may be difficult to obtain a sense orunderstanding of the surrounding environment. It may prove difficult toobtain an environmental awareness regarding, e.g., what is below theaircraft, given a depth of field from a distance.

When considering an optionally piloted vehicle (OPV), an unmanned aerialvehicle (UAV) with no crew aboard, or a piloted aircraft with blindzones, the difficulty encountered in terms of depth of field andsituational awareness may play a significant role. For example, havingan understanding of the depth of field may play a role during aircraftoperations, such as load pickup or drop-off and aircraft landing.

In order to maneuver an aircraft, remote operations have relied onjoystick or other control input mechanisms. The operator adjusts adirection of the aircraft, monitors the status of the aircraft's flight,and then adjusts the direction again in order to have the aircraftarrive at a destination. Such techniques result in frequent engagementby the operator, are prone to operator/human error, susceptible tolatency, are time consuming, are taxing on an operator, and imposeincreased stress on the aircraft based on quick or rapid changes interms of movement.

BRIEF SUMMARY

An embodiment is directed to a method comprising receiving, by a controlstation, an input including a command for re-location of a vehicle froma first location to a second location, the input identifying the secondlocation, determining, by the control station, at least one of adistance, a direction, an altitude, and a latitude and longitude for thevehicle to travel from the first location to the second location, andtransmitting, by the control station, the command and the at least oneof a distance, a direction, an altitude, and a latitude and longitude tothe vehicle.

An embodiment is directed to an apparatus comprising at least oneprocessor, and memory storing instructions that, when executed by the atleast one processor, cause the apparatus to: receive an input includinga command for re-location of an aircraft from a first location to asecond location, the input identifying the second location, determine atleast one of a distance, a direction, and a latitude and longitude forthe aircraft to travel from the first location to the second location,and transmit the command and the at least one of a distance, adirection, and a latitude and longitude to the aircraft.

An embodiment is directed to a system comprising a control stationcomprising a touchscreen configured to receive a one-touch input thatincludes a command for re-location of at least one of an optionallypiloted vehicle (OPV) and an unmanned aerial vehicle (UAV) from a firstlocation to a second location and identifies the second location,determine at least one of a distance and new latitude and longitude forthe at least one of an OPV and a UAV to travel from the first locationto the second location, and transmit the command and the at least one ofa distance and new latitude and longitude.

Other embodiments are described below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example and not limitedin the accompanying figures in which like reference numerals indicatesimilar elements.

FIG. 1 illustrates an environment in accordance with one or more aspectsof this disclosure;

FIGS. 2A-2D illustrate exemplary windows in accordance with one or moreaspects of this disclosure;

FIG. 3 illustrates exemplary parameters for calculating a distance inaccordance with one or more aspects of this disclosure; and

FIG. 4 illustrates an exemplary method in accordance with one or moreaspects of this disclosure.

DETAILED DESCRIPTION

In accordance with various aspects of the disclosure, apparatuses,systems and methods are described for enhancing the operation of anaircraft. In some embodiments, operation may be enhanced by providing anoperator a vantage point from directly below the aircraft. As describedherein, touch positioning may eliminate guess work for depth perception,allowing an operator to confidently position the aircraft over or awayfrom objects on the ground. While largely described in connection withaircraft (e.g., airplanes, helicopters, etc.), the techniques andmethodologies described herein may be adapted to accommodate other formsor types of vehicles. For example, the techniques and methodologies maybe adapted to accommodate operations associated with marine vessels(e.g., boats, ships, yachts, submarines, etc.), automobiles (cars,trucks, etc.), etc.

It is noted that various connections are set forth between elements inthe following description and in the drawings (the contents of which areincluded in this disclosure by way of reference). It is noted that theseconnections in general and, unless specified otherwise, may be direct orindirect and that this specification is not intended to be limiting inthis respect. In this regard, a coupling of entities may refer to eithera direct or an indirect connection.

FIG. 1 illustrates an environment 100 in accordance with one or moreaspects of this disclosure. The environment 100 may include an aircraft10. In some embodiments, the aircraft 10 may be, or include, anoptionally piloted vehicle (OPV) or an unmanned aerial vehicle (UAV).The OPV or UAV may be configured to conduct one or more tasks ormissions, such as loading or unloading supplies, equipment, etc. Theaircraft 10 may include one or more components to collect or recordinformation or data related to flight. For example, the aircraft 10 mayinclude a vehicle management system (VMS) 12 that may be configured tocollect flight status data from one or more sensors 14 included in theaircraft 10. In some embodiments, the sensors 14 include a sensor (e.g.,camera 14-1 shown in FIG. 3).

The environment 100 may include a ground control station (GCS) 16 or analternative control device. The GCS 16 may be configured to communicatewith the aircraft 10. The GCS 16 may communicate commands or directivesto the aircraft 10 to control the operation(s) of the aircraft 10. Forexample, the GCS 16 may command or direct the aircraft 10 based onparameters or criteria associated with a task or mission, based on thedata collected by the VMS 12, or based on any other input factor orcondition. The GCS 16 may be, or include, a server computer, a desktopcomputer, a laptop computer, a mobile device (e.g., a smartphone orcellphone), a PDA, MFD, etc.

The GCS 16 and the aircraft 10 may be coupled to one another andconfigured to communicate with one another via a communication device18. The communication device 18 may be, or include, one or more links(e.g., data links), routers, access points, communication media, etc.The communication device 18 may be configured to communicate with theaircraft 10 and/or the GCS 16 in accordance with one or morecommunication types, standards, or protocols. Communications between twoor more entities may be encrypted to provide privacy or secrecy.

The environment 100 is illustrative. In some embodiments, additionalentities not shown in FIG. 1 may be included. In some embodiments, oneor more of the entities (or portions thereof) may be optional. Forexample, in some embodiments the aircraft 10 may communicate directlywith the GCS 16, such that the communication device 18 might not beincluded, or the communication device 18 may be subsumed in the GCS 16and/or the aircraft 10. In some embodiments, the GCS 16 may be includedin, or reside on, the aircraft 10.

FIGS. 2A-2D (collectively referred to as FIG. 2) illustrate windows thatmay be made available to an operator, such as an operator of the GCS 16of FIG. 1. The windows may have been captured by one or more sensors,such as a video camera mounted on the belly or underside of the aircraft10 directly over a cargo hook of the aircraft 10. The windows may havebeen communicated to the GCS 16 from the aircraft 10 (potentially by wayof the communication device 18) in, e.g., real-time to provide a livefeed from the aircraft 10 to the GCS 16.

As shown in the window of FIG. 2A, the aircraft 10 may initially belocated at, or proximate to, a point 202 in window of FIG. 2A. The point202 may correspond to a loiter point, or a location of hover of theaircraft 10. The point 202 may correspond to a center of a ring 204. Thering 204 may provide an operator with a perspective of distance, muchlike a scale may be used in connection with a map. In some embodiments,a radius of the ring 206, which may be measured in, e.g., feet, may bedisplayed in the window to facilitate such perspective.

Also shown in FIG. 2A is a point 208. The point 208 may correspond to alocation associated with a task or mission. For example, the point 208may correspond to a loading (e.g., pickup) or unloading (e.g., drop-off)destination for one or more goods or products, or a landing point forthe aircraft 10.

An operator of the GCS 16 may command the aircraft 10 to travel ornavigate from the point 202 to the point 208. For example, as shown inFIG. 2B, the operator of the GCS 16 may depress or touch a portion of atouchscreen associated with the GCS 16 coinciding with the location ofthe point 208. An icon or other indicia may be displayed or overlaid onthe window to signify to the operator that the command was accepted bythe GCS 16. Color-coding may be used to indicate various stages ofoperation of the aircraft 10. For example, an icon 210 shown in FIG. 2Bmay take on a first color, such as a white color or appearance, tosignify to the operator of the GCS 16 that the re-location command wasreceived by the GCS 16 but has not yet been accepted by the aircraft 10.

Following receipt of the command at the GCS 16 to re-locate orre-position the aircraft 10, the GCS 16 may transmit the command to theaircraft 10, along with an identification of the desired new locationfor the aircraft (e.g., the location coinciding with point 208). Oncethe command and the new location are received by the aircraft 10 (e.g.,by the VMS 12 or a flight control computer), the aircraft 10 mayauthenticate the command to confirm that the communication was receivedfrom a trusted source (e.g., from the GCS 16) or is a valid command. Theaircraft 10 may transmit an acknowledgment message to the GCS 16 uponreceipt of the command or upon the authentication of the command. Uponreceipt of the acknowledgment message, the GCS 16 may change the colorof the icon 210 from the first color (e.g., white) to a second color(e.g., magenta) to signify to the operator of the GCS 16 that theaircraft 10 has accepted the instruction to navigate to the new location(e.g., towards point 208).

FIG. 2C illustrates a window reflecting the progress of the navigationof the aircraft 10 to the point 208. Relative to FIGS. 2A-2B, as shownin FIG. 2C, the ring 204 encloses or encompasses the point 208/icon 210,which may provide the operator of the GCS 16 with confidence that theaircraft 10 is moving in the appropriate direction (e.g., in thedirection of the point 208). Moreover, the icon 210 may continue to bemagenta in color in FIG. 2C to symbolize that the re-location command isactive or still in progress.

FIG. 2D illustrates a window indicative of the aircraft 10 havingreached the point 208 (or a point that is proximate to point 208 withinan acceptable threshold). For example, the point 208/icon 210 may beapproximately centered with respect to the ring 204. In connection withthe aircraft 10 having arrived at the new location, the icon 210 maychange color to signify that the re-location operation is complete andthat the aircraft 10 is awaiting further commands. For example, inconnection with the aircraft 10 having arrived at the new location asindicated in FIG. 2D, the icon 210 may change from the second color(e.g., magenta) back to the first color (e.g., white) or to a thirdcolor (e.g., green).

As shown in FIG. 3, the aircraft 10 may include a sensor 14-1. Thesensor 14-1 may be downward-oriented, such that the footage (e.g., videoor image(s)) captured by the camera 14-1 may be indicative of the groundbelow the aircraft 10.

The sensor 14-1 may have a known or predetermined field of view (FOV)equal to two times the parameter labeled ‘a’ (reference character 302)in FIG. 3, where ‘a’ may correspond to an angle. The aircraft 10 (e.g.,the camera 14-1) may be located at a known height ‘h’ (referencecharacter 304) above the ground. Based on the height ‘h’ (304), a radius‘r’ (reference character 306) of the FOV may be calculated. For example,using geometry the radius ‘r’ (306) may be calculated as the product of:(1) the height ‘h’ (304), and (2) the tangent of the angle ‘a’ (302).Expressed as a formula, the calculation may correspond to: r=h×tan(a).

In FIG. 3, a parameter labeled ‘P’ (reference character 308) mayrepresent a percentage (e.g., a fixed percentage) of the radius ‘r’(306) corresponding to a location ‘d’ (reference character 310) of a newdestination (e.g., point 208). Thus, taking the product of ‘P’ (308) and‘r’ (306) will yield the distance ‘d’ (310) to the new location (208) asmeasured from an origin ‘0.0’. Expressed as a formula, d=P×r. The origin‘0.0’ in FIG. 3 may correspond to the (initial) point 202 of FIG. 2.

The calculation described above in connection with FIG. 3 to acquire thevalue or parameter ‘d’ may be conducted any number of times. Forexample, if a Cartesian coordinate system (e.g., X and Y axes) is used,d_(x) and d_(y) subcomponents of the ‘d’ value may be calculated in turnand then summed (e.g., via vector summation) to obtain ‘d’. Othercalculations may be performed. For example, a translation may beperformed in a ‘z’ direction, wherein instead of ‘d’ being a percentageof ‘r’, dZ may be a factor of P times a fixed vertical distance.

In some embodiments, rather than simply using a coordinate system (suchas a Cartesian coordinate system) to define the new location ‘d’, adirectional or compass distance may be used. For example, the d_(x) andd_(y) subcomponents may be converted to, e.g., distance east anddistance north based on a (north-easterly) direction in which theaircraft 10 is heading.

Still further, directional or compass distance, in potential combinationwith an initial position (e.g., point 202 of FIG. 2) of the aircraft 10,may be converted into coordinates for the new location of the aircraft10 (e.g., point 208 of FIG. 2). As an example, if the initial position(e.g., point 202 of FIG. 2) of the aircraft 10 corresponds to latitude 1(lat₁) and longitude 1 (lon₁), and the new or desired position (e.g.,point 208 of FIG. 2) of the aircraft 10 corresponds to latitude 2 (lat₂)and longitude 2 (lon₂), lat₂ and lon₂ may be calculated as follows:lat₂ =a sin(sin(lat₁)*cos(d/R)+cos(lat₁)*sin(d/R)*cos(θ))lon₂=lon₁ +a tan 2(sin(θ)*sin(d/R)*cos(lat₁),cos(d/R)−sin(lat₁)*sin(lat₂))

where θ may be the bearing (in radians, measured from a referencedirection (e.g., clockwise from north)) and d/R may be the angulardistance (potentially measured in terms of radians), where d may be thedistance travelled and R may be the earth's radius (where the Earth'smean radius may be equal to 6,371 km).

One or more of the conversions, calculations, or computations describedabove may be performed by one or more entities. For example, one or moreof the calculations or computations may be performed by the aircraft 10,the GCS 16, the communication device 18, or any other component ordevice.

FIG. 4 illustrates a method in accordance with one or more aspects ofthis disclosure. The method may be used in connection with one or moreapparatuses, systems, devices, or components, such as those describedherein. The method of FIG. 4 may be used to direct an aircraft from afirst location to a second location.

In step 402, the aircraft 10 may transmit data to one or more entitiesor control devices, such as GCS 16. The data may include sensor data,such as camera images or video, altitude (e.g., parameter ‘h’ (304) ofFIG. 3), a specification of a current direction of travel of theaircraft 10, and current coordinates (e.g., latitude and/or longitudecoordinates) of the aircraft 10.

In step 404, the control device may cause one or more parameters orsensor data associated with the aircraft 10 to be presented. Forexample, the control device may display one or more of the video or theimage received from the aircraft 10, the altitude of the aircraft 10, acurrent hover point or location of the aircraft 10 (e.g., point 202 ofFIG. 2), and a ring (e.g., ring 204 of FIG. 2).

In step 406, the control device may receive an input. The input maycorrespond to a command from a user or operator to re-locate theaircraft 10 from the first location to a second location. The input maybe based on activation or actuation of a touchscreen, in response to avoice command, etc. As part of step 406, the control device may cause anicon (e.g., icon 210 of FIG. 2) or other indicator to be displayed orplayed (e.g., an audio indication) that indicates that the controldevice received the input.

In step 408, the input command of step 406 may be transformed into acommanded location, such as a distance ‘d’ (e.g., distance ‘d’ (310) ofFIG. 3). The distance ‘d’ may be specified in accordance with, e.g., acoordinate system (e.g., a Cartesian coordinate system), a directionalor compass distance (e.g., distance west and distance south), or aspecification of coordinates (e.g., latitude and longitude).

In step 410, a specification of the input command of step 406 along withthe commanded location of step 408 may be communicated from the controldevice to the aircraft 10 via a message.

In step 412, the aircraft 10 may receive the message of step 410. Inresponse to receiving the message, the aircraft 10 may begin to navigateto the desired or commanded (second) location (e.g., point 208 of FIG.2). As part of step 412, the aircraft 10 may transmit an acknowledgmentof receipt of message to an operator or the control device.

In step 414, the control device may receive the acknowledgment of step412 and modify or alter the indicator based on the receipt. For example,if an icon is used, as part of step 414 the control device may change acolor of the icon from a first color (e.g., white) to a second color(e.g., magenta) in response to receiving the acknowledgment message fromthe aircraft 10.

In step 416, a determination may be made whether the aircraft 10 iswithin a threshold of a commanded location. For example, the controldevice may monitor the location of the aircraft 10 as it progresses fromthe first location (e.g., point 202 of FIG. 2) to the second location(e.g., point 208 of FIG. 2). If the aircraft 10 has not arrived at thesecond location (“No” path out of step 416), flow may remain at step 416to continue monitoring the status of the aircraft's navigation, or flowmay proceed (back) to step 412 as shown in FIG. 4. If, on the otherhand, the aircraft 10 has arrived at the second location (“Yes” path outof step 416), flow may proceed from step 416 to step 418.

In step 418, the aircraft 10 may have arrived at the second location. Insome embodiments, the control device may receive an acknowledgement orcompletion message as part of step 418. In response to determining thatthe aircraft 10 arrived at the second location, the control device maymodify or alter the icon or indicator to, e.g., a third color (e.g.,green). The method may end at step 418, wherein the control device mayawait additional commands or inputs in connection with step 406.

The method of FIG. 4 is illustrative. In some embodiments, some of thesteps (or portions thereof) may be optional. Additional steps not shownmay be included. In some embodiments, the method may execute in an orderor sequence different from what is shown in FIG. 4. For example, steps402 and 404 may execute more than once or continuously over the courseof the method in order to provide the operator with updated or real-timedata as to the progress of the aircraft's navigation from the firstlocation to the second location.

As described herein, sensor data taken from an aircraft may be presentedto an operator. The sensor data may include infrared or optical imagery,synthetic or still images, Light Detection And Ranging (LIDAR) data,Laser Detection and Ranging (LADAR) data, etc.

As described herein, in some embodiments various functions or acts maytake place at a given location and/or in connection with the operationof one or more apparatuses, systems, or devices. For example, in someembodiments, a portion of a given function or act may be performed at afirst device or location, and the remainder of the function or act maybe performed at one or more additional devices or locations.

Embodiments of the disclosure may be implemented using one or moretechnologies. In some embodiments, an apparatus or system may includeone or more processors, sensors, and memory storing instructions that,when executed by the one or more processors, cause the apparatus orsystem to perform one or more methodological acts as described herein.Various mechanical components known to those of skill in the art may beused in some embodiments.

Embodiments of the disclosure may be implemented as one or moreapparatuses, systems, and/or methods. In some embodiments, instructionsmay be stored on one or more computer-readable media, such as atransitory and/or non-transitory computer-readable medium. Theinstructions, when executed, may cause an entity (e.g., an apparatus orsystem) to perform one or more methodological acts as described herein.In some embodiments, the functionality described herein may beimplemented in hardware, software, firmware, or any combination thereof.

Embodiments of the disclosure may be tied to one or more particularmachines. For example, a control station may receive one or more inputs.The input may specify an action to be taken by a vehicle, such as anaircraft. For example, the input may serve as a command that directs theaircraft to re-locate from a first location to a second location. Theinput/command may be transmitted to the aircraft, such that the aircraftis directed to the second location. In some embodiments, one-touchpositioning in connection with a touchscreen may be used to command anaircraft to re-locate.

Embodiments of the disclosure may transform an article into a differentstate or thing. For example, sensor data may be transformed into acurrent or first location for an aircraft. The sensor data may bepresented to an operator, and the operator may be able to select anoperation (e.g., a re-location operation for the aircraft from the firstlocation to a second location) based on the sensor data.

Aspects of the disclosure have been described in terms of illustrativeembodiments thereof. Numerous other embodiments, modifications andvariations within the scope and spirit of the appended claims will occurto persons of ordinary skill in the art from a review of thisdisclosure. For example, one of ordinary skill in the art willappreciate that the steps described in conjunction with the illustrativefigures may be performed in other than the recited order, and that oneor more steps illustrated may be optional in accordance with aspects ofthe disclosure.

What is claimed is:
 1. An apparatus comprising: a touchscreen; at leastone processor; and memory storing instructions that, when executed bythe at least one processor, cause the apparatus to: receive an input onthe touchscreen including a command for re-location of an aircraft froma first location to a second location, the input identifying the secondlocation, determine at least one of a distance, a direction, and alatitude and longitude for the aircraft to travel from the firstlocation to the second location, and transmit the command and the atleast one of a distance, a direction, and a latitude and longitude tothe aircraft; wherein the touchscreen is configured to display a mapcomprising an identification of the first location enclosed by a ringand a scale that provides a distance associated with the ring, the ringrepresenting a radius on the ground, the radius varying with variationsin altitude of the aircraft based on an angle of a field of view of acamera on the aircraft and the altitude of the aircraft.
 2. Theapparatus of claim 1, wherein the apparatus comprises a mobile deviceconfigured as a control station.
 3. The apparatus of claim 1, whereinthe instructions, when executed by the at least one processor, cause theapparatus to: determine a distance for the aircraft to travel from thefirst location to the second location, wherein the distance is specifiedby the apparatus in terms of at least one of: a Cartesian coordinatesystem, a directional or compass distance, and latitude and longitudecoordinates.
 4. The apparatus of claim 1, wherein: the touchscreen isconfigured to display a map overlaid with data received from sensors ofthe aircraft, the data comprising an identification of the firstlocation.
 5. The apparatus of claim 4, wherein the instructions, whenexecuted by the at least one processor, cause the apparatus to: receivethe input via a depression of a point on the touchscreen, wherein thepoint corresponds to the second location.
 6. The apparatus of claim 1,wherein the instructions, when executed by the at least one processor,cause the apparatus to: determine a distance for the aircraft to travelfrom the first location to the second location, and calculate thedistance as a percentage of the radius.
 7. The apparatus of claim 1,wherein the instructions, when executed by the at least one processor,cause the apparatus to: determine a distance for the aircraft to travelfrom the first location to the second location, receive anacknowledgment of the transmission of the command and the distance, andchange the state of an indicator presented at the apparatus from a firststate to a second state responsive to the received acknowledgment. 8.The apparatus of claim 7, wherein the indicator comprises an icondisplayed on a display screen of the apparatus, and wherein theinstructions, when executed by the at least one processor, cause theapparatus to: receive an acknowledgement that the aircraft arrives atthe second location, and change the state of the indicator to a thirdstate responsive to receiving the acknowledgment that the aircraftarrives at the second location.
 9. A system comprising: a controlstation comprising a touchscreen configured to receive a one-touch inputthat includes a command for re-location of at least one of an optionallypiloted vehicle (OPV) and an unmanned aerial vehicle (UAV) from a firstlocation to a second location and identifies the second location,determine at least one of a distance and new latitude and longitude forthe at least one of an OPV and a UAV to travel from the first locationto the second location, and transmit the command and the at least one ofa distance and new latitude and longitude; wherein the touchscreen isconfigured to display a map comprising an identification of the firstlocation enclosed by a ring and a scale that provides a distanceassociated with the ring, and wherein the second location is associatedwith at least one of: a loading or an unloading destination for one ormore goods or products or a landing point for the at least one of an OPVand a UAV, and wherein at least a portion of an indicator overlaid ontop of the map is configured to change from a first color to a secondcolor responsive to the control station receiving an acknowledgment ofthe transmission of the command and the at least one of a distance andnew latitude and longitude, and wherein at least a portion of theindicator is configured to change from the second color to a third colorresponsive to acknowledgment that the at least one of an OPV and a UAVarrives at the second location.
 10. The system of claim 9, furthercomprising: a communication device configured to receive the command andthe at least one of a distance and new latitude and longitude from thecontrol station and transmit the command and the at least one of adistance and new latitude and longitude to the at least one of an OPVand a UAV.
 11. The system of claim 9, wherein the control stationcomprises a control device configured to display data received from oneor more sensors of the at least one of an OPV and a UAV, the datacomprising a camera image or video captured by the at least one of anOPV and a UAV, altitude of the at least one of an OPV and a UAV, aspecification of a current direction of travel of the at least one of anOPV and a UAV, and latitude and longitude coordinates of the at leastone of an OPV and a UAV when at the first location.